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Abstract 

Since 1997, the identity of the research field of adults learning mathematics has been debated; 
the research field has grown in quantity and quality; and the research forum Adults Learning 
Mathematics (ALM) has established an international journal. In practice, the researchers answer 
the question about identity and quality of research papers in committees, in editorial teams or as 
referees in journals. The purpose of this article is to create a starting point for a debate, based on 
quality criteria in the field of mathematics education research, on specific criteria to locate 
quality of research papers on practice-related educational research in the field of adults learning 
mathematics. Following general criteria in mathematics education any research paper has to 
explain and present its own problematique explicitly i.e. define the problem field and the 
theoretical/methodological/philosophical approach of the study reported. Additional criteria 
specific to the field of adults learning mathematics are suggested, namely, the author should 
position themselves by answering the following three questions: (1) What is adult mathematical 
knowledge (or adult numeracy)? (2) How do adults learn mathematics? (3) Why teach 
mathematics to adults? 
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A search for identity and quality 

For the last 15 years a new international research field has been cultivated in the borderland 
between mathematics education and adult education. Conceptual frameworks and theoretical 
approaches are imported from the two neighbouring fields and restructured within the new 
community of practice and research (Wedege, 2001). The international research forum Adults 
Learning Mathematics (ALM) was created in 1994 and need was felt for an identity debate 
within this new research area. Thus, the following question was fonnulated at the Fourth 
International Conference on Adults Learning Mathematics, ALM4, in 1997: “Could there be a 
specific problematique for research in adult mathematics education?” (Wedege, 1998). The 
debate continued at the following conferences and it was reported in the proceedings (e.g. 
Wedege, Benn & Maasz, 1999). Coben (2000) presented the identity debate in the introduction, 
“Perspectives on research on adults learning mathematics”, of the first international anthology 
from the field, and FitzSimons, Coben and O’Donoghue (2003) examined the current state of 
research in adult mathematics in the chapter “Lifelong mathematics education” of the Second 
International Handbook on Mathematics Education. In a review of research on adult numeracy, 
Coben (2003) also presented the debate on adult mathematics education as a research domain. 

In this debate, the focus is on common characteristics in the construction of the area of 
investigation (adult numeracy and mathematics) and in the need for multi- and inter-disciplinary 
inquiry (pedagogy, psychology, sociology, anthropology etc.). Mertens (2005) presents the 
paradigm in educational research as “a way of looking at the world [...] composed of certain 
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philosophical assumptions that guide and direct thinking and action” (p. 7). According to her 
analysis, the central issues in a debate on identity of a specific “paradigm” in education research 
are ontological, epistemological and methodological. Where ontological issues deal with the 
nature of reality, epistemological issues with the nature of knowledge and with the relation 
between knower and would-be-known, and methodological issues deal with the approach to 
systematic enquiry. This notion of paradigm is consistent with an epistemological framework 
which I construed and called the concept of problematique in mathematics education with 
inspiration in French epistemology. Problematique covers a systematically linked problem - 
and practice - field within mathematics education research. Researchers working within a given 
problematique have a specific theoretical and/or methodological approach to the subject area 
(e.g. adults learning mathematics), and their approach is defined by ontological, epistemological 
and methodological choices (see Wedege, 1998, 2001, 2006). In the following section, I shall 
further present the terminology of problematique. 

Within the research field of adult mathematics education, we may find different 
problematiques. Even incommensurable problematiques occur, like the one found in the 
International Adult Literacy Survey (OECD, 1995) - presuming that it is meaningful to measure 
people’s quantitative literacy with the same tool across different countries and cultures - versus 
investigations from an ethnomathematical perspective, which emphasizes the local and the 
culturally specific (Rogoff, 1984). In both cases, the subject area is people’s mathematical 
everyday competences, but the subject field is construed differently because of different 
research interests and questions, theories and methodologies. However, I have claimed that the 
problematiques represented within the international research forum of ALM have a series of 
common and specific traits. Among these are the following three characteristics (Wedege, 
2001 ). 

1. Subject area'. The learner is in focus, and their “ numeracy” is understood as mathematical 
knowledge. 

Adult numeracy is the main construct in the subject field. In English speaking countries, the 
tenn “numeracy” is used for certain basic skills and understandings in mathematics which 
people need in various situations in their daily life. Numeracy is a key word in basic adult 
mathematics education. As a concept, however, numeracy is deeply contested in politics, 
education and research. Nevertheless, as an analytical concept, adult numeracy may be 
considered as mathematical activity in its cultural and historical context. For a review of 
international research and related literature on adult numeracy, see Coben (2003). 

2. Two approaches: The duality between the objective and subjective perspective is implicit, 
or explicit, in all studies. 

Two different lines of approach are possible and intertwined in the research: the objective 
approach, starting either with societal and labour market requirements with regard to adults’ 
mathematics-containing competences or with demands from the academic discipline 
(transformed into “school mathematics”), versus the subjective approach starting with 
adults’ need for mathematics-containing competences and their beliefs and attitudes towards 
mathematics (Wedege, 2001). 

3. Justification: The general aim of education and of research is “empowerment” of adults 
learning mathematics. 

This statement was concluded in the debate of “Adults Learning Mathematics as a 
community of practice and research” (Wedege, Benn & Maasz, 1999). Later Johansen 
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(2006) has drawn the same conclusion based on her study of the justification problem from 

a discourse analytical perspective. 

At the Thirteenth International Conference of Adults Learning Mathematics, in 2006, I 
fonnulated this question: “Could there be specific criteria for quality of research papers in the 
field of adults learning mathematics?” It was obvious for me that this debate was needed as a 
next step in the search for identity. And indeed, at the conference the quality debate was 
welcomed by the participants. My purpose was to create a starting point for a debate on specific 
quality criteria within our field on the basis of general criteria for educational research and more 
specific criteria in the field of mathematics education. 

Problematiques in mathematics education 

In educational sciences and mathematics education research literature dealing with general 
issues of identity and quality, the term “paradigm” is used often with a broad meaning like in 
Mertens (2005), Furlong and Oancea (2005) and Dorfler (1993). According to Dorfler (1993), 
there exist different, even mutual exclusive paradigms, in the community of classical research in 
mathematics education. One of them for instance is the strong orientation to mathematical content, 
another is the social-constructivist paradigm. However, in mathematics education, the term 
“paradigm” has also been connected with Kuhn’s concept of paradigm defined as a system of 
common interests and presumptions characterising “normal science” in a specific historical 
period, a so-called “disciplinary matrix” (Kuhn, 1962). In this sense it is used to state that 
mathematics education research is weak in paradigms (see, for example, Niss, 2007). However, 
paradigms in the sense that Kuhn used them in sciences (mainly physics) do not exist in 
humanities or social sciences where several competing and incommensurable theories 
legitimately exist side by side at any time. 

Like paradigm, the French term “ problematique ” is also used in a broad sense e.g. in 
Dorfler (1993) and in Sierpenska and Kilpatrick (1998). I have argued that the meaning of 
“problematique” in mathematics education is “problem field within which the coherence is defined 
by a science (psychology, sociology, anthropology etc.) and/or a theory” (Wedege, 2006, p. 321). 
In order to reconnoitre the complex scientific landscape of the borderland between mathematics 
education and adult education, I have used an epistemological tenninology with the key terms: 
subject area, subject field, problem field and problematique as analytical tools (Wedege, 1998, 
2001,2006). 

In any research process the subject area (the area to be investigated), is decided upon and a 
simple structure is fonnulated. Subjects within the area of adults learning mathematics might be, 
for example, the following phenomena: 

• mathematics in adult vocational training; 

• basic mathematics in the Brazilian adult education; 

• adults’ views of mathematics; or 

• learning mathematics as part of lifelong learning. 

Taking the point of departure in a specific position, the researcher then adopts a certain view of the 
subject and identifies a problem field concerning the subject area by formulating problem 
complexes, hi this way, the subject area is further structured into a subject field, the field which is 
actually investigated. Problem complexes might be, for example: 

• the tension between the constraints and the needs felt by adults to learn mathematics; 

• the neglect of developing adults’ statistical literacy in adult education; or 

• the identification of adults’ mathematical understandings with common sense. 

Within the community of ALM and other communities of research, problem complexes are 
fonnulated in the fonn of research questions and answers about the subject field on the basis of a 
specific theoretical and/or methodological approach, and a systematically linked problem field a 
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problematique is constructed. The subject field is opened and closed at the same time during this 
process. New questions, which could not be posed before the conceptualization, are formulated and 
as the complexity of the problem field is reduced other questions cannot be formulated. For 
example, questions concerning possible contributions from mathematics education to the 
development of adults’ technological competences in the workplace presume a technology concept 
(Wedege, 2000). The Danish proverb, “Som man raber i skoven far man svar” (generally translated 
to: “You get what you’re asking for”) represents one of the basic ideas in this concept of 
problematique. Another epistemological presumption is that problems fonnulated as research 
questions are the driving force of the research (Bachelard, 1949). 

Why do we need quality criteria? 

The existence of criteria, no matter how provisional or incomplete, allows researchers to assess the 
quality of their work or the work of others, and it allows the field to see what progress, if any, is 
being made. [...] the criteria are lenses through which the research landscape can be viewed. 
(Kilpatrick, 1993, p. 31) 

Any scientific discipline has its specific quality criteria. Hence, quality was naturally one of the 
dimensions to be debated in the search for identity of mathematics education as a research 
domain, at the ICMI study conference in 1994 (Sierpinslca & Kilpatrick, 1998). The question 
discussed in the working group was: “What criteria should be used to evaluate the results of 
research in mathematics education?” The result of the discussion was a framework for 
examining criteria for the quality of research in mathematics education, and a series of questions 
for continuing the debate was fonnulated. One question was, “Could the relation of mathematics 
education to classroom practice generate criteria for quality in research in mathematics 
education?” Three of the papers presented at the conference dealt with quality (Hanna, 1998; 
Hart, 1998; Lester & Lambdin, 1998). 

The issue of quality has not been on the agenda explicitly during the ALM conference 
sessions. In practice, the researchers answer the question about identity and quality of research 
papers in committees, in editorial teams or as referees in journals. According to the former chief 
editor, FitzSimons (2006) of Adults Learning Mathematics - an International Journal (ALMIJ), 
the “refereed electronic journal has now been established to further develop the high quality 
work in this field” (p. 246). She presented and discussed the technical and conceptual 
requirements for the preparation of articles for the refereed journal, at the ALM 12 conference in 
2005. Instead of the timid question “could there be specific criteria for quality in the field of 
adults learning mathematics” the question might be fonnulated as: “What are the specific 
criteria ...?” 


Structuring and restricting the debate 

As the issue of this article is specific quality criteria for research papers in the field of ALM, I 
have to clarify the meaning of the two tenns “research” and “specific”. 

In clarifying research I follow Zan (2004) in her conception of research as “disciplined 
inquiry”, meaning that in order to be “research” a study in mathematics education has to possess 
some characteristics, for example that: 

• the study must be intentional enquiry, aimed to face a specific problem; 

• the study must be connected with theory; 

• the study must be connected with mathematics educational practice; 

• the study must be public and verifiable; and 
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• the research procedures ought to be checkable. 

This tentative answer to the question “What is research?” leads to the question “What is good 
research?”. Zan (2004) has proposed a list of quality criteria: objectivity, validity, generality, 
relevance, utility, ethics, originality, replicability etc. But a central point in her paper is that 
there is no agreement among researchers about these criteria and that the researchers’ 
epistemological choices play an important role in the choice of quality criteria. 

In clarifying “specific” we need to consider - specific in relation to what? In my work, I 
lean on the debate within educational research and within mathematics education. As mentioned 
above, any scientific discipline has its own quality criteria. There exist general criteria for 
educational sciences. There are quality criteria specific to mathematics education. Furthermore, 
there are criteria for specific fonns of theoretical and empirical research in education, for 
example fundamental research and action research. In order to assess quality in applied and 
practice-based educational research, Furlong and Oancea (2005) propose four dimensions and 
sub-dimensions of quality as presented in Table 1. I have chosen this example to illustrate the 
complexity of the quality debate in educational sciences, which also encompasses economic, 
technological and political issues. 

Table 1. Dimensions of quality in applied and practice-based educational research (Furlong and 
Oancea, 2005, p. 15) 


Epistemic: 
Methodological and 
theoretical robustness 

Technological 

Capacity 
development and 
value for people 

Economic 

Trustworthiness 

Purposivity 

Plausibility 

Marketability and 
competitiveness 

Builds on what is 

known + contribution 
to knowledge 

Salience/ timeliness 

Partnership, 
collaboration and 
engagement 

Cost-effectiveness 

Explicitness 

Specificity and 
accessibility 

Reflexivity, 
deliberation and 

criticism 

Auditability 

Propriety 

Concern for enabling 
impact 

Receptiveness 

Feasibility 

Paradigm-dependent 

criteria 

Flexibility and 
operationalisability 

Transformation and 
personal growth 

Originality 

Scientific robustness 

Social & economic robustness 



The four dimensions of quality, in the analysis of Furlong and Oancea (2005), are the epistemic 
dimension, the technological dimension, the capacity development dimension and the economic 
dimension. In this paper, I restrict the discussion to epistemic questions concerning 
“methodological and theoretical robustness”. The five sub-dimensions of the epistemic 
dimension are trustworthiness; builds on what is known plus contribution to knowledge; 
explicitness; propriety; and paradigm dependent criteria. By paradigm they mean “a complex of 
epistemological/philosophical and methodological traditions, shared practices etc. used within a 
particular “epistemic community” (Furlong & Oancea, 2005, p. 12), and they state that criteria 
may well vary depending on the particular paradigm used. With this broad meaning of the tenn 
“paradigm” - also compatible with the concept of problematique -1 claim with reference to my 
analysis of the field (Wedege, 2001) that the core of the international research on adults learning 
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mathematics is dominated by a specific paradigm - or type of problemcitique - and thus are to 
be evaluated by similar quality criteria. 

Research versus report 

When a referee comments on a research paper it is always necessary to assess both the research 
behind the paper AND the paper as a research report. The research being satisfactory is a 
necessary but not sufficient condition for a research report to be accepted. In tenns of the report, 
there are particular criteria, e.g. location and discussion of the author’s contribution to the 
research field and epistemological and methodological consciousness and reflection that need to 
be considered. Such criteria concern the quality of the conducted research. In order to structure 
the quality debate I suggest to distinguish analytically between: 

• quality of the conducted research; and 

• quality of the paper as a report of the research and its results. 

In the literature on quality in mathematics education, we find this distinction in that some of the 
articles concern the research (Hart, 1997; Kilpatrick, 1993; Vital & Valero, 2003; Zan, 2004), 
some the research papers (reports or dissertations) (Dorfler, 1993; Hanna, 1998, FitzSimons, 
2006) and some both (Lester & Lambdin, 1998). 

The focus of this article is the report - not the conducted research - and I shall reduce the 
complexity of the quality debate further. The research field of ALM is situated in the borderland 
between mathematics and adult education (Wedege, 2001) and as such it is opened and closed 
by the two large fields. However, in my discussion, I lurther restrict the problem of quality 
criteria in two ways: (a) I look at research papers through quality criteria fonnulated within 
mathematics education research, and (b) I look for criteria for papers dealing with practice- 
related educational research - meaning that the research behind the report addresses a specific 
problem in the practice of adult mathematics education and not an epistemological, 
methodological or a purely theoretical problem. 

General quality criteria for research papers in mathematics education 

At a symposium in Demnark on criteria for scientific quality and relevance in the didactics of 
mathematics, Dorfler (1993), fonner editor of Journal fiir Matematikdidaktik and of 
Educational Studies in Mathematics, presented a list of requirements or demands, which he used 
to obtain a rather formal assessment of a research paper. By virtue of Dorfler’s position in the 
field, these criteria might be considered as representing the first step towards what I seek: a 
checklist for any author of scientific articles in mathematics education. Here follows my 
summary of the list: 

1. There should be an explicitly formulated rationale for the presented research: What 
are the goals? What is the motivation? The central research questions? 

2. The research paradigm, the background philosophy should be made explicit and 
recognizable. 

3. The employed research method and research design must be clearly stated and 
described — especially when it is about empirical research. 

4. In the case of theoretical papers or sections, assertions, statements and theses must 
be argued and made plausible within the assumed premises and framework. 

5. In empirical research a clear and orderly presentation of the results is essential and 
the results must be strictly separated from their interpretation. 
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6. A general requirement is for a reasonable embeddedness in existing research and 
literature. Research is a social process and this should be reflected in every single 
paper to a certain extent. 

7. The author must make plausible to the reader the relevance of the research to 
mathematics education. Not that the results could necessarily be applied in the 
classroom the next day, but in some way the paper should be concerned with 
teaching/leaming mathematics. 

8. Almost trivial are requirements like: clear and well structured, well defined or 
explicated terms and concepts, relatedness of the various parts of a paper, 
systematic referencing etc. (from Dorfler, 1993, pp. 85-87). 

These criteria are relevant to all kind of mathematics education research, not only practice 
related research. However, all the points 1-8 concern the epistemic dimension of Table 1, and 
only one of the requirements (7), is specific to mathematics teaching and learning. But the 
criteria have been developed and formulated in the practice of two prominent journals in 
mathematics education research and, according to Dorfler (1993), “the specificity of 
mathematics education research derives from the range of human experience it studies and 
organizes” (p.79). 

In (1) Dorfler claims the importance of making the research interest and questions 
visible, and in (2) he talks about the “research paradigm” (or “background” philosophy) which 
also has to be explicit. In (7), he states that the problems and the results have to be relevant to 
mathematics education. The three central criteria together are consistent with the concept of 
problematique where the meaning of the problem in mathematics education is seen as the 
driving and defining force in research (Wedege, 2001). Thus, it is possible in my work on 
specific quality criteria in ALM at the same time to use the terminology of problematique and to 
build on the general criteria fonnulated by Dorfler. 

Specific quality criteria within the field of adults learning mathematics? 

In FitzSimons’s (2006) presentation at ALM12, the question of quality of an article for Adults 
Learning Mathematics - an International Journal (ALMIJ) was discussed in general tenns; e.g. 
Does it make an original contribution to mathematics education for adults? And does it provide 
a well founded and cogently argued analysis?. But she also put forward a specific requirement: 
“Articles must be relevant to adult mathematics/ numeracy education (...)”. Here the criteria of 
relevance is defined in relation to research and theoretical perspectives; to debate; and to 
practice in the area of adults learning mathematics/ numeracy (FitzSimons, 2006 p. 246). 
Another specific criterion from the referee report of ALMIJ is that the submitted article should 
clearly be a study in the mathematics education of adults. FitzSimons presents the criterion like 
this: 


... it is important to make the links to adults learning mathematics explicit. One of the problems 
faced by adult educators, internationally, is the definition of ‘adult’. Accordingly, it may be useful 
for the author/s to define the kinds of learners for whom the article is intended to be of relevance 
(e.g., early school leavers, older workers needing retraining, parents wishing to help their children, 
or new immigrants and others wishing to return to study/employment). Another ongoing debate is 
over the definition of terms such as ‘mathematics’ versus ‘numeracy’ or ‘mathematical literacy’, 
and so forth. Authors could briefly clarify their choice of terminology, explaining why they have 
chosen the particular term/s and what might be included as content or learning outcomes 

(Op citp. 247). 

In other words, the author must position themselves in the research field of adults learning 
mathematics. Given that we have a long series of international journals in mathematics 
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education it is important that any author submitting an article to ALMIJ makes explicit their 
reasons for this particular choice of journal. At a first glance the journal will publish articles 
concerned with “all aspects of adult mathematics/numeracy education”. A second glance shows 
that the conception of adult mathematics education is - and has to be - open to discussion and 
negotiation. 

In the organising team of Topic Study Group 6 (TSG6), Adult and lifelong mathematics 
education, at the 10 th International Congress on Mathematics Education (ICME) in 2004, we 
decided the following limitation - and opening - of the subject area of adult mathematics 
education in our call for papers: 

By adults we mean people with the identity of citizens, workers, parents, un-employed, engineers 
etc. The term lifelong indicates that education takes place in all facets and spheres of life. By 
mathematics we mean multiple activities and knowledge, including academic mathematics, 
vocational mathematics, ethnomathematics, folk mathematics and adult numeracy. Regarding 
education we have adopted the terminology of UNESCO (2000) as a point of departure” with a 
distinction between informal, formal and non-formal education 

(from “Aims and focus”, TSG6, ICME-10, 2004). 

In TSG6 discussions, the issues were restricted to those following. 

• Adult numeracy as a competence, building a bridge between school and personal, civic 
and working life. 

• Adults’ beliefs, attitudes and emotions to mathematics, including their resistance and 
motivation to learn mathematics. 

• The role of technology in adult lifelong mathematics education (informal, fonnal and 
non-formal). 

• Global aspects, such as the role of large-scale studies of adults’ mathematical “needs”. 

And the TSG6 concluded that: “Adult and lifelong mathematics education has multiple 
dimensions and we have to approach this subject area from psychological, sociological, 
anthropological, linguistic, philosophical, economic and political perspectives. The studies and 
experiences are likely to be linked with issues of class, gender and race” (from “Aims and 
focus”, TSG6, 2004). 

Thus, it seems like the specificity of criteria for research papers and articles in ALM so far 
concerns the definition/limitation of the subject area in the study reported (what are the 
phenomena being investigated) and the opening of a problem field, for example the bridging of 
school and every day life, and adults’ resistance to learn mathematics. I propose that the next 
step in the quality work is a demand for clarification of the problematique within which the 
study is undertaken. The paper should explain and present its own problematique — i.e. answer 
the following questions: 

• What is the problem in the practice of adults learning mathematics and what are the 
central research questions of the study? 

• Why is the problem being investigated - what is the purpose of the study? 

• What is the theoretical/philosophical framework of the study? 

• What is the methodological approach? 

When the author gives an answer to these questions about the background, goals, framework 
and approach, at the same time he/she should respond to the following three questions, 
implicitly or explicitly: 

1. What is adult mathematical knowledge (numeracy)? 

2. How do adults learn mathematics? 


Volume 4(1) - February 2009 


13 




Tine Wedege: Specific quality criteria for research papers on adults learning mathematics? 


3. Why teach mathematics to adults? 

In this way, the reader will have infonnation about the author’s conception of and construction 
of the subject field (1), about the learning theory behind the study (2), and about the author’s 
answer to the justification problem, which is closely related to the “what” and the “how” 
questions in mathematics education (3). Like this the author will present her/his position in 
research based on certain interest and values. 

Provisional conclusion 

The purpose of this article is to present and discuss quality criteria for papers dealing with 
practice-related educational research on adults learning mathematics; i.e. where the underlying 
research addresses a specific problem in the practice of adult mathematics education and not an 
epistemological, methodological or a purely theoretical problem. As we have seen, in 
mathematics education it is a general demand that any research paper should explain and present 
its own problematique, i.e. background, purpose, research questions, theoretical framework and 
methodology. On this basis, I suggest that the specific quality criteria for papers dealing with 
practice-related educational research in adults learning mathematics as formulated in the last 
section, be taken as the conclusion of this paper and, hopefully, the beginning of the further 
debate in the international research community of adults learning mathematics. 
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